The devices used by people to interact with computers have dramatically changed over the past few decades. As the speed and processing power of computers have increased, the devices and methods for interaction between man and computer have improved. Voluntary input devices such as the mouse, joystick, touch pad, touch screen, and keyboard have been developed to make computers easier to use.
In 1972, Pong.TM., one of the first video computer games, was introduced. In that game, a simulated ball would "bounce" over a line between opposite sides of a monitor screen, as a tennis ball bounces over a net from one side of the court to the other. A paddle, analogous to a tennis racquet, was controlled by the player and used to direct the "ball" from the player's side of the screen to the opposite side of the screen. If the ball passed the player's paddle, the player would lose the point. The Pong.TM. game entertained thousands of people and helped begin the video game revolution.
Video computer games of many different kinds are now available, both for arcade and home play. Video computer games can be categorized as fighting, adventure, role playing, puzzle, sporting, racing, and simulation games. This list of categories is not intended to be exhaustive. Other categories and categorization schemes may exist.
Fighting games are characterized by a one-on-one contest between the player's character and another character. The other character can be controlled by either the computer or a second player. The object is to win the contest.
In an adventure game, the player's character is on a journey through a graphical world where he is confronted by other characters and obstacles. Points are awarded to the player for various actions, such as killing an evil character, retrieving an item, or reaching a goal. The object is to achieve the highest point score.
In a role playing game, the player is on an imaginary journey and encounters numerous obstacles, such as evil characters, collapsed bridges, quicksand, trap doors, and the like. In order to remain in the game, the player must overcome these obstacles. The object of the game is to complete the journey.
The player in a puzzle game must solve a puzzle. The difficulty of the game may be increased by imposing time constraints, increasing the difficulty of the puzzle itself, or by imposing some other limitation. The goal is to solve the puzzle within the given constraints.
In sporting games, the player controls simulated athletes or equipment in a sporting event, such as a football, baseball, hockey, or basketball game. The dexterity and strength of the simulated athletes, or the behavior of the equipment, such as a golf club and golf ball, is programmed into the game. The player's object is to win the sporting event.
Racing games are a hybrid of sporting games and adventure games. The player in a racing game navigates a vehicle in a race or on a mission. The object of the game is to finish the race or mission before any other competitor or in the shortest time.
Simulation games mimic the experience of operating an actual vehicle such as an aircraft, tank, or submarine. The object of the simulation is to master control of the vehicle while attempting to destroy an enemy or complete an obstacle course.
In existing video computer games, the computer's output is based solely on the voluntary responses of a user through a voluntary input device such as a joystick. Since video computer games respond in the same manner when given the same input, video computer games lack variability. Often players find a video computer game trite after playing it numerous times and memorizing the appropriate inputs necessary to achieve the desired result.
Computers are not used only to play video computer games, of course. A large and growing number of people use the computer to communicate with others. E-mail and the World Wide Web are available to millions of people around the world. Interactive games, quiz games, mind games, and games of truth can be played over computer networks and the Internet.
Recently, "chat rooms" and other computer conferencing systems have become increasingly popular. Chat rooms permit computer users to communicate over the Internet, an on-line service, or other computer network, by displaying typed messages, sound clips, and video images as they are entered by each user in the chat room. Chat rooms provide a forum for discussing subjects such as business, sex, theater, hobbies, and sports. Chat rooms are an increasingly popular form of entertainment.
The goal of video computer games and computer conferencing systems is to excite, entertain, and impart information to the user. However, current video computer games and computer conferencing systems have no way of determining whether the user is indeed excited or entertained by the activity, because video computer games and other forms of computer entertainment utilize only the user's voluntary responses. In one video computer game, for example, good eye-hand coordination or the ability to run in place on a pressure sensitive pad while controlling a joystick in response to visual and auditory signals may be the only inputs required. In another video computer game, the user's problem solving ability may be all that is required. Since users can easily memorize the voluntary inputs needed to win a video computer game and the video computer game responds the same way each time to a given voluntary input, the video computer game becomes predictable and, ultimately, boring.
While all video computer games and computer conferencing systems require intentional and voluntary inputs from the user, the emotional state of the user remains undetected and unused. In order for the user to express his emotional state, the user must perform a voluntary act. Currently, users attempt to convey their emotions to other computer users by using various symbols (e.g., the symbol ":-)" represents a smile). In all these activities, the user is usually limited to input from a keyboard, mouse, microphone, video camera, or other voluntary input device. However, the actual emotional state of the user is never directly input to the computer.
By restricting the input of the computer to voluntary acts by the user, the ability of the computer to be used as a means of communication is greatly limited. While communicating over a computer, a user has no means to communicate his emotional state based upon actual physiological or autonomic nervous system responses while communicating other information voluntarily. Similarly, a user receiving information has no way of "sensing" the emotional state of the other computer user.
Computers have been used to collect data about the autonomic responses of a subject in the context of medical monitoring and treatment. In U.S. Pat. No. 5,441,047, an ambulatory patient health monitoring system is disclosed where a patient is monitored by a health care worker at a central station while the patient is at a remote location. Various items of medical condition sensing and monitoring equipment are placed in the patient's home, depending on the particular medical needs of the patient. The patient's medical condition is sensed and measured in the home, and the data are collected by a computer and transmitted to the central station for analysis and display. The health care worker then is placed into interactive visual communication with the patient so that the health care worker can assess the patient's general well being as well as the patient's medical condition.
In another medical application, signals from a patient's heart are fed to a computer for analysis and generation of a display indicative of the patient's heart rate. The heart rate information is presented to the patient. Then, the patient concentrates on the heart rate information display in an attempt to lower his heart rate. This type of "bio-feedback" is a clinical tool that is commonly used to teach patients to control certain of their autonomic functions.
Computers have also used physiological data to control a simulation game. U.S. Pat. No. 5,470,081 discloses a golf simulator which monitors brain waves to control the flight of a simulated golf ball. If the monitored brain waves suggest a high level of concentration, the simulator causes the ball to fly straight. If the monitored brain waves suggest excitement, and thus a lower degree of concentration, the simulator causes the ball to hook or slice. The monitored brain waves exclusively control the flight of the ball, and the flight of the ball is not responsive to any voluntary inputs from the player.
U.S. Pat. No. 4,358,118 discloses a quiz game which uses a physiological response. A computer measures the user's skin resistance in response to a posed question. Then, the user's skin resistance, which indicates to the user how he is reacting to the question, is displayed by the computer. The user then voluntarily enters a response to the question using the computer keyboard based on the displayed skin resistance. The computer selects the next question based on his answer to the previous question and thus guides the user through a programmed series of questions. The computer responds solely to the user's voluntary answer to the question.
Physiological responses are also used in communications contexts. In U.S. Pat. No. 5,047,952, a communication system using an instrumented glove is disclosed for deaf, deaf and blind, or non-vocal individuals. Strain gage sensors in the glove detect movements of the user's hand. The movements detected by the sensors are transmitted to a computer which translates the movements into letters, words, or phrases. The output devices for communicating depend on the visual, vocal, and hearing capabilities of the individuals and can be selected from a voice synthesizer, LCD monitor, or Braille display. The computer responds only to the user's voluntary inputs.
As all of these examples illustrate, prior to the present invention computers have not used a combination of user voluntary and autonomic responses to control a computer system to provide a more realistic game experience or more complete communication of information.
The present invention utilizes both a user's voluntary actions and the user's autonomic nervous system responses as an indicator of emotions to allow for more intimate interaction with other computer users in computer conferencing systems and for more engaging and exciting simulators, video computer games, entertaiunent programs, and other interactive programs. The detection of the user's voluntary actions and autonomic nervous system responses enables the computer to respond to both the user's emotional state and voluntary actions.